Transmitting device and receiving device

ABSTRACT

A transmitting device (30, 30a) is configured to transmit, to a receiving device (40, 40a), a plurality of video signals captured from different positions, the plurality of video signals being grouped by a plurality of groups depending on imaging positions at which the video signals are captured. The transmitting device (30, 30a) comprises: a controller (32) configured to assign an ID for identifying each of the plurality of groups; and a communication interface (37) configured to transmit a video signal to which the ID is assigned, to the receiving device (40, 40a).

CROSS REFERENCE TO RELATED APPLICATION

This application claims priority to and the benefit of Japanese PatentApplications No. 2019-239564 and No. 2019-239570 filed on Dec. 27, 2019,the entire disclosures of which are incorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to a transmitting device that transmits aplurality of video signals captured from different positions and areceiving device that receives the video signals.

BACKGROUND

As advanced types of 360-degree video, video systems that enable usersto view 360-degree videos from desired viewpoints are studied. Forexample, NPL 1 describes use cases such as “3D Image Messaging”involving viewpoint movements in the range in which a user in a seatedposition moves his or her head as 3 DoF+ (degree of freedom) and“Immersive 6 DoF Streaming” involving viewpoint movements in the rangein which a user moves freely as 6 DoF.

Typically, one camera captures video from one viewpoint. The use of aplurality of videos (multiview video) captured by a plurality of camerasmakes it possible to generate video from an intermediate positionbetween the video capturing positions (i.e. imaging positions). This isa known technology called viewpoint interpolation, intermediateviewpoint image generation, etc. (for example, see PTL 1)NPL 2 definesthe omnidirectional media format (OMAF) as a file format for storing,for example, 360-degree video signals of predetermined viewpoints.

CITATION LIST Patent Literature

PTL 1: JP 6306952 B2

Non-Patent Literature

NPL 1: 3GPP TR 26.928, “3rd Generation Partnership Project; TechnicalSpecification Group SA WG4 Extended Reality in 5G; (Release 16)”, ver0.5.0, 2019-07

NPL 2: ISO/IEC 23090-2: 2019, “Information technology—Codedrepresentation of immersive media—Part 2: Omnidirectional media format”

SUMMARY Technical Problem

In viewpoint interpolation, a plurality of video signals captured fromdifferent positions are used to generate video from a desired viewpoint.On the other hand, for service that enables viewing video from theviewpoint of a freely moving user as in 6 DoF, it is necessary tocapture a plurality of videos centering on many locations and performviewpoint interpolation. However, the captured video signals are signalsindependent of each other, and there is no information for associatingthese information with each other. OMAF supports storing video signalsof fixed viewpoints, but does not support processing a plurality ofvideo signals in the case where the viewpoint is freely moved. Thus,there is no method for a receiving terminal to specify what kinds ofvideo signals are present and how these video signals relate to eachother, and no method for a receiving terminal to, in the case ofreceiving a plurality of video signals, specify how to associate thesevideo signals.

It could therefore be helpful to provide a transmitting device and areceiving device that can easily specify video signals necessary forviewpoint interpolation in the case where a user changes his or herviewpoint position.

Solution to Problem

A transmitting device according to an embodiment is a transmittingdevice configured to transmit, to a receiving device, a plurality ofvideo signals captured from different positions, the plurality of videosignals being grouped by a plurality of groups depending on imagingpositions at which the video signals are captured, the transmittingdevice comprising: a controller configured to assign an ID foridentifying each of the plurality of groups; and a communicationinterface configured to transmit a video signal to which the ID isassigned, to the receiving device.

In an embodiment, the controller may be configured to specify a requestID that is an ID of a video signal corresponding to a change of aviewpoint position of a user of the receiving device, and thecommunication interface may be configured to transmit a video signal towhich the request ID is assigned, to the receiving device.

In an embodiment, the communication interface may be configured to:transmit a plurality of video signals necessary to generate videocorresponding to the viewpoint position of the user in a video signalgroup to which the request ID is assigned, in the case where the requestID is same as an ID of an immediately previously transmitted videosignal; and transmit a default video signal in the video signal group towhich the request ID is assigned, in the case where the request ID isdifferent from the ID of the immediately previously transmitted videosignal.

In an embodiment, the plurality of video signals may be grouped by theplurality of groups of a plurality of levels depending on the imagingpositions, and the controller may be configured to assign a group ID foridentifying each of the plurality of groups.

In an embodiment, a first-level group out of the plurality of groups ofthe plurality of levels may be a group to which a plurality of videosignals used to generate video from a viewpoint different from animaging position in the receiving device belong.

In an embodiment, a second-level group out of the plurality of groups ofthe plurality of levels may be one or more first-level groups to which aplurality of video signals captured in a same location belong.

In an embodiment, the plurality of video signals belonging to thefirst-level group may be compressed by predictive coding using acorrelation between the video signals, when transmitted by thecommunication interface.

In an embodiment, the communication interface may be configured tosimultaneously transmit all of the plurality of video signals to each ofwhich the ID is assigned, to the receiving device.

In an embodiment, the plurality of video signals may be grouped by theplurality of groups depending on the imaging positions, and thecontroller may be configured to assign a group ID for identifying eachof the plurality of groups.

In an embodiment, a first-level group out of the plurality of groups maybe a group to which a plurality of video signals used to generate videofrom a viewpoint different from an imaging position in the receivingdevice belong.

In an embodiment, a second-level group out of the plurality of groupsmay be one or more first-level groups to which video signals captured ina same location belong.

In an embodiment, the plurality of video signals belonging to thefirst-level group may be compressed by predictive coding using acorrelation between the video signals, when transmitted by thecommunication interface.

In an embodiment, the ID and the video signal may be stored in a file ofISO base media file format.

A receiving device according to an embodiment is a receiving deviceconfigured to receive a video signal transmitted from the foregoingtransmitting device, the receiving device comprising: a controllerconfigured to decode the video signal to generate decoded video, andacquire a viewpoint position of a user; and a communication interfaceconfigured to transmit viewpoint information indicating the viewpointposition of the user, to the transmitting device.

A receiving device according to an embodiment is a receiving deviceconfigured to receive a video signal transmitted from the foregoingtransmitting device, the receiving device comprising a controllerconfigured to decode the video signal to generate decoded video, andacquire a viewpoint position of a user, wherein the controller isconfigured to specify respective position information of imaging devicesthat capture a plurality of video signals necessary to generate videocorresponding to the viewpoint position of the user, and performviewpoint interpolation using decoded video corresponding to theposition information.

Advantageous Effect

It is thus possible to easily specify video signals necessary forviewpoint interpolation in the case where a user changes his or herviewpoint position.

BRIEF DESCRIPTION OF THE DRAWINGS

In the accompanying drawings:

FIG. 1 is a block diagram illustrating an example of a videotransmission system according to Embodiment 1;

FIG. 2 is a diagram explaining first group IDs according to Embodiment1;

FIG. 3 is a diagram explaining second group IDs according to Embodiment1;

FIG. 4 is a diagram explaining a hierarchical structure of video signalsaccording to Embodiment 1;

FIG. 5A is a diagram illustrating an example of the structure of a filestoring video signals according to Embodiment 1;

FIG. 5B is a diagram illustrating an example of the structure of a filestoring video signals according to Embodiment 1;

FIG. 6 is a diagram illustrating an example of display by a receivingdevice according to Embodiment 1;

FIG. 7 is a sequence diagram illustrating an example of the operation ofthe video transmission system according to Embodiment 1,

FIG. 8 is a block diagram illustrating an example of a videotransmission system according to Embodiment 2;

FIG. 9 is a sequence diagram illustrating an example of the operation ofthe video transmission system according to Embodiment 2;

FIG. 10 is a block diagram illustrating an example of a videotransmission system according to Embodiment 3;

FIG. 11 is a sequence diagram illustrating an example of the operationof the video transmission system according to Embodiment 3; and

FIG. 12 is a block diagram illustrating the schematic structure of acomputer.

DETAILED DESCRIPTION

Disclosed embodiments will be described in detail below, with referenceto the drawings.

Embodiment 1

(Video Transmission System)

First, a video transmission system according to Embodiment 1 will bedescribed below. FIG. 1 is a diagram illustrating a video transmissionsystem 1 according to Embodiment 1, and schematically illustratestransmission of a plurality of video signals captured in a plurality oflocations. The video transmission system 1 includes a plurality ofimaging devices (cameras) 10, a plurality of first transmitting devices20, a second transmitting device 30, and a receiving device 40. In thisspecification, the term “location” denotes a predetermined area providedfor a specific purpose, such as a venue, a stadium, a facility, or apark. In this embodiment, the video transmission system 1 includes threefirst transmitting devices 20. A first transmitting device 20A transmitsvideo signals captured in a location A, a first transmitting device 20Btransmits video signals captured in a location B, and a firsttransmitting device 20C transmits video signals captured in a locationC.

Each imaging device 10 assigns metadata to a video signal captured at apredetermined position and transmits the video signal with the metadatato the corresponding first transmitting device 20. Herein, the metadatatransmitted by the imaging device 10 is position information indicatingthe imaging position of the imaging device 10, and may includeinformation indicating the imaging direction. The metadata may includedistance information indicating the distance from the imaging device 10to the captured object.

In detail, a plurality of imaging devices 10A installed in the locationA each assign metadata to a video signal captured at a predeterminedposition in the location A and transmit the video signal with themetadata to the first transmitting device 20A. A plurality of imagingdevices 10B installed in the location B each assign metadata to a videosignal captured at a predetermined position in the location B andtransmit the video signal with the metadata to the first transmittingdevice 20B. A plurality of imaging devices 10C installed in the locationC each assign metadata to a video signal captured at a predeterminedposition in the location C and transmit the video signal with themetadata to the first transmitting device 20C.

The first transmitting device 20A receives the video signals and themetadata from the plurality of imaging devices 10A, assigns IDs to thevideo signals, and transmits them to the second transmitting device 30.The first transmitting device 20B receives the video signals and themetadata from the plurality of imaging devices 10B, assigns IDs to thevideo signals, and transmits them to the second transmitting device 30.The first transmitting device 20C receives the video signals and themetadata from the plurality of imaging devices 10C, assigns IDs to thevideo signals, and transmits them to the second transmitting device 30.

Transmission from the imaging devices 10 to the first transmittingdevices 20 and transmission from the first transmitting devices 20 tothe second transmitting device 30 may be performed using the SerialDigital Interface (SDI), the Internet Protocol (IP), or the like. Morespecifically, in the case of using SDI, transmission may be performed by12G-SDI, 3G-SDI, HD-SDI, or the like. In the case of using IP,transmission may be performed by the Real-time Transport Protocol (RTP)based on the SMPTE (Society of Motion Picture and Television Engineers)ST 2110 standard. For example, in the case of transmitting a videosignal without encoding it, SDI or IP may be used. In the case oftransmitting a video signal after encoding it, IP may be used. In thisembodiment, each imaging device 10 transmits a video signal to thecorresponding first transmitting device 20 without encoding it, and thefirst transmitting device 20 transmits the video signal to the secondtransmitting device 30 after encoding it. Alternatively, each imagingdevice 10 may perform encoding, and each first transmitting device 20may not perform encoding.

The second transmitting device 30 aggregates video signals transmittedfrom the plurality of first transmitting devices 20, and transmits thevideo signals to the receiving device 40 via a transmission path (forexample, IP network). The video signals need not necessarily beaggregated in the second transmitting device 30, and the firsttransmitting devices 20 may transmit the video signals directly to thereceiving device 40 via any of various transmission paths such as abroadcast transmission path, a terrestrial IMT (International MobileTelecommunication) network, an optical fiber, and a Wi-Fi® (Wi-Fi is aregistered trademark in Japan, other countries, or both) line.

The receiving device 40 may be any device capable of receiving videotransmission from the first transmitting devices 20 or the secondtransmitting device 30. Examples of the receiving device 40 include ahead-mounted display, virtual reality (VR) goggles, a tablet terminal, asmartphone, and a personal computer (PC).

(First Transmitting Device)

The first transmitting devices 20 according to Embodiment 1 will bedescribed below.

Each first transmitting device 20 includes a video receiver 21, a videoencoder 22, a first ID assigner 23, a storage 24, and a videotransmitter 25, as illustrated in FIG. 1 . The video encoder 22 and thefirst ID assigner 23 are included in a controller. The controller may beimplemented by dedicated hardware such as an application specificintegrated circuit (ASIC) or a field-programmable gate array (FPGA),implemented by a processor, or implemented by a combination of dedicatedhardware and a processor. The video receiver 21 and the videotransmitter 25 are included in a communication interface.

The video receiver 21 acquires video signals and metadata from theplurality of imaging devices 10, and outputs them to the video encoder22.

The video encoder 22 encodes the video signals received from the videoreceiver 21 according to any video coding method such as H.265/HEVC(High Efficiency Video Coding), to generate the encoded video signals.The video encoder 22 may perform compression by predictive coding usingcorrelation between video signals, for efficient transmission. The videoencoder 22 outputs the encoded video signals to the storage 24.

In the present disclosure, the video signals are grouped by a pluralityof groups depending on the imaging positions. The plurality of videosignals captured by the plurality of imaging devices 10 can be groupedas video signal groups each used to generate video from a viewpointposition different from an imaging position (i.e. used for viewpointinterpolation). This grouping is referred to as “first-level grouping”to distinguish it from the below-described grouping. That is, eachfirst-level group is a group to which a plurality of video signals usedto generate video from a viewpoint different from an imaging position inthe receiving device 40 belong.

The first ID assigner 23 inputs IDs (first group IDs) for identifyingthe respective first-level groups according to operation by an operatorof the first transmitting device 20, and stores the first group IDs andthe video signals in the storage 24 in association with each other. Thatis, the first ID assigner 23 assigns the first group IDs to the videosignals.

The first group IDs will be described below, with reference to FIG. 2 .FIG. 2 illustrates an example of capturing video at a baseball stadium.A total of 15 imaging devices 10, i.e. five imaging devices 10 in thehorizontal direction and three imaging devices 10 in the verticaldirection, are installed on the behind the catcher side of the baseballstadium. The respective position information p of these 15 imagingdevices 10 are denoted as 1 to 15, for the sake of convenience. A totalof 15 imaging devices 10, i.e. five imaging devices 10 in the horizontaldirection and three imaging devices 10 in the vertical direction, areinstalled on the first base side of the baseball stadium. The respectiveposition information p of these 15 imaging devices 10 are denoted as 16to 30, for the sake of convenience. A total of 15 imaging devices 10,i.e. five imaging devices 10 in the horizontal direction and threeimaging devices 10 in the vertical direction, are installed on the thirdbase side of the baseball stadium. The respective position information pof these 15 imaging devices 10 are denoted as 31 to 45, for the sake ofconvenience. Although each imaging device 10 captures only forward videofrom the imaging position in the example illustrated in FIG. 2 , theimaging device 10 may capture 360-degree surround video from the imagingposition.

The receiving device 40 receives video signals captured by the imagingdevices 10 of position information p=1 to 15, and performs viewpointinterpolation using the received video signals. Thus, the receivingdevice 40 can present, to a user, video of viewpoints in a certain rangefrom the behind the catcher side. For example, in the case where thereceiving device 40 is displaying a video signal captured by the imagingdevice 10 of position information p=8 and subsequently the user moveswithin a predetermined range (for example, within a radius of a fewmeters), the receiving device 40 can present, to the user, videocorresponding to the viewpoint of the user. Likewise, in the case wherethe receiving device 40 receives video signals captured by the imagingdevices 10 of position information p=16 to 30, the receiving device 40can present, to the user, video of viewpoints in a certain range fromthe first base side. In the case where the receiving device 40 receivesvideo signals captured by the imaging devices 10 of position informationp=31 to 45, the receiving device 40 can present, to the user, video ofviewpoints in a certain range from the third base side.

For example, in the case where video signals are captured by the imagingdevices 10 of position information p=1 to 45, the first ID assigner 23assigns “I” as a first group ID to the 15 video signals captured fromthe behind the catcher side, assigns “II” as a first group ID to the 15video signals captured from the first base side, and assigns “III” as afirst group ID to the 15 video signals captured from the third baseside, as illustrated in FIG. 2 . The first ID assigner 23 may assign afirst group ID to every video signal. In the case where the videoencoder 22 performs compression using correlation between video signals,the first ID assigner 23 may assign a first group ID only to eachreference video signal (for example, video signals captured by theimaging devices 10 of position information p=8, 23, 38).

Referring back to FIG. 1 , the storage 24 stores the encoded videosignals and the metadata (position information) received from the videoencoder 22, and the metadata (information indicating first group ID)received from the first ID assigner 23.

The video transmitter 25 transmits the encoded video signals and themetadata (position information and information indicating first groupID) stored in the storage 24, to the second transmitting device 30.

Each imaging device 10 and the whole or part of the corresponding firsttransmitting device 20 may be integrated. For example, in the case wherethe imaging device 10 has the function of the video encoder 22, thevideo receiver 21 in the first transmitting device 20 receives anencoded video signal from the imaging device 10 and outputs it to thestorage 24.

(Second Transmitting Device)

The second transmitting device 30 according to Embodiment 1 will bedescribed below.

The second transmitting device 30 includes a video receiver 31, a secondID assigner 32, a storage 33, a viewing request receiver 34, a viewpointinformation receiver 35, a position determiner 36, and a videotransmitter 37, as illustrated in FIG. 1 . The second ID assigner 32 andthe position determiner 36 are included in a controller. The controllermay be implemented by dedicated hardware such as an ASIC or a FPGA,implemented by a processor, or implemented by a combination of dedicatedhardware and a processor. The video receiver 31, the viewing requestreceiver 34, the viewpoint information receiver 35, and the videotransmitter 37 are included in a communication interface.

The video receiver 31 receives encoded video signals and metadatatransmitted from the plurality of first transmitting devices 20, andoutputs the received encoded video signals and metadata to the storage33.

The video signals captured by the plurality of imaging devices 10 can begrouped as video signal groups each of which is a group of video signalscaptured in the same location. This grouping is referred to as“second-level grouping” to distinguish it from the above-describedfirst-level grouping. That is, each second-level group is one or morefirst-level groups to which a plurality of video signals captured in thesame location belong.

The second ID assigner 32 inputs IDs (second group IDs) for identifyingthe respective second-level groups according to operation by an operatorof the second transmitting device 30, and stores the second group IDsand the video signals in the storage 33 in association with each other.That is, the second ID assigner 32 assigns the second group IDs to thevideo signals.

The second group IDs will be described below, with reference to FIG. 3 .In the example illustrated in FIG. 3 , the second transmitting device 30receives video signals captured in the location A from the firsttransmitting device 20A. Specifically, the second transmitting device 30receives 15 video signals of first group ID=1 captured by the imagingdevices 10 of position information p=1 to 15, 15 video signals of firstgroup ID=II captured by the imaging devices 10 of position informationp=16 to 30, and 15 video signals of first group ID=III captured by theimaging devices 10 of position information p=31 to 45. The secondtransmitting device 30 also receives video signals captured in thelocation B from the first transmitting device 20B. Specifically, thesecond transmitting device 30 receives, from the first transmittingdevice 20B, 15 video signals of first group ID=I captured by the imagingdevices 10 of position information p=46 to 60, 15 video signals of firstgroup ID=II captured by the imaging devices 10 of position informationp=61 to 75, and 15 video signals of first group ID=III captured by theimaging devices 10 of position information p=76 to 90. The secondtransmitting device 30 also receives video signals captured in thelocation C from the first transmitting device 20C. Specifically, thesecond transmitting device 30 receives, from the first transmittingdevice 20C, 15 video signals of first group ID=I captured by the imagingdevices 10 of position information p=91 to 105, 15 video signals offirst group ID=II captured by the imaging devices 10 of positioninformation p=106 to 20, and 15 video signals of first group ID=IIIcaptured by the imaging devices 10 of position information p=121 to 135.

For example, the second ID assigner 32 assigns “A” as a second group IDto the 45 video signals captured in the location A, assigns “B” as asecond group ID to the 45 video signals captured in the location B, andassigns “C” as a second group ID to the 45 video signals captured in thelocation C.

FIG. 4 illustrates a hierarchical structure in which video signals aregrouped at two levels in the above-described manner. Suppose 45 videosignals are captured in each of the locations A, B, and C, asillustrated in FIG. 3 . In this case, the 45 video signals captured inthe location A are classified as the group A of second group ID=A, andfurther classified, in the group A, into the 15 video signals belongingto first group ID=I, the 15 video signals belonging to first groupID=II, and the 15 video signals belonging to first group ID=III. The 45video signals captured in the location B are classified as the group Bof second group ID=B, and further classified, in the group B, into the15 video signals belonging to first group ID=I, the 15 video signalsbelonging to first group ID=II, and the 15 video signals belonging tofirst group ID=III. The 45 video signals captured in the location C areclassified as the group C of second group ID=C, and further classified,in the group C, into the 15 video signals belonging to first group ID=I,the 15 video signals belonging to first group ID=II, the 15 videosignals belonging to first group ID=III.

Referring back to FIG. 1 , the storage 33 stores the encoded videosignals and the metadata received from the video receiver 31, and themetadata received from the second ID assigner 32.

The viewing request receiver 34 receives a viewing request indicatingcontent to be viewed from the receiving device 40, and outputs theviewing request to the video transmitter 37.

The viewpoint information receiver 35 receives viewpoint informationindicating the viewpoint position of the user of the receiving device 40(for example, the position of the receiving device 40) from thereceiving device 40, and outputs the viewpoint information to theposition determiner 36. For example, the viewpoint position informationis three-dimensional coordinates in a global coordinate system.Alternatively, the viewpoint position information may bethree-dimensional coordinates in a local coordinate system relative to adefault user position.

The position determiner 36, having received the viewpoint informationfrom the viewpoint information receiver 35, determines whether a changeof the viewpoint position of the user involves a change of ID (firstgroup ID and second group ID). Specifically, the position determiner 36specifies a request ID which is the ID of a video signal correspondingto the change of the viewpoint position of the user, and determineswhether the request ID is the same as the ID of the immediatelypreviously transmitted video signal.

In the case where the request ID is the same as the ID of theimmediately previously transmitted video signal, the position determiner36 outputs the respective position information of a plurality of imagingdevices 10 that capture a plurality of video signals necessary togenerate video corresponding to the viewpoint position of the user in avideo signal group to which the request ID is assigned, to the videotransmitter 37. For example, the position determiner 36 determines animaging position corresponding to the viewpoint position of the user,and outputs, to the video transmitter 37, the respective positioninformation of the imaging devices 10 located adjacent to or surroundingthe imaging position.

In the case where the request ID is different from the ID of theimmediately previously transmitted video signal, the position determiner36 outputs the position information of an imaging device 10 thatcaptures a default video signal in the video signal group to which therequest ID is assigned, to the video transmitter 37.

The video transmitter 37 transmits one or more video signals to thereceiving device 40. In this specification, “to transmit a video signal”means to transmit a video signal and metadata assigned to (i.e.multiplexed with) the video signal simultaneously. The video transmitter37 may transmit only the metadata. Video signals belonging to afirst-level group and transmitted by the video transmitter 37 may becompressed by predictive coding using correlation between video signalsby the video encoder 22 in the first transmitting device 20.

The video transmitter 37, having received the viewing request from theviewing request receiver 34, acquires, from the storage 33, a defaultvideo signal of the content requested for viewing, all ID information(information indicating first group ID and second group ID) relating tothe content requested for viewing, and the position information of theimaging device 10 representative of each first-level group, andtransmits them to the receiving device 40. The receiving device 40presents which viewpoint positions are selectable to the user, based onthe ID information and the position information of the imaging device10.

The video transmitter 37, having received the position information fromthe position determiner 36, acquires a video signal captured by theimaging device 10 having the position information (i.e. a video signalcorresponding to the position information) from the storage 33, andtransmits it to the receiving device 40. In detail, in the case wherethe request ID is the same as the ID of the immediately previouslytransmitted video signal, the video transmitter 37 transmits theplurality of video signals necessary to generate the video correspondingto the viewpoint position of the user in the video signal group to whichthe request ID is assigned. In the case where the request ID isdifferent from the ID of the immediately previously transmitted videosignal, the video transmitter 37 transmits the default video signal inthe video signal group to which the request ID is assigned.

FIGS. 5A and 5B are each a diagram illustrating an example of a filestructure in the case of storing IDs and video signals in a file of ISObase media file format such as OMAF. The basic unit of the format iscalled “box”.

In the file structure illustrated in FIG. 5A, when storing n videosignals having a common second group ID and first group ID in a file,the n video signals are stored under boxes indicating the second groupID and the first group ID, thus indicating the group to which the videosignals belong. In the file structure illustrated in FIG. 5B, whenstoring n video signals having a common second group ID and first groupID in a file, a track storing each video signal is stored together withboxes indicating the second group ID and the first group ID, thusindicating the group to which the video signals belong.

Although two-level grouping is described in this embodiment, grouping atthree or more levels may be performed depending on content according tooperation by the user. In the case of grouping at three or more levels,too, each video signal can be identified through hierarchization. Forexample, one or more second-level groups to which video signals capturedin the same region belong may be grouped as a third-level group.

(Receiving Device)

The receiving device 40 according to Embodiment 1 will be describedbelow.

The receiving device 40 includes a viewing request transmitter 41, avideo receiver 42, a video decoder 43, a storage 44, a viewpointposition detector 45, a viewpoint information transmitter 46, aviewpoint interpolator 47, an operation information adder 48, and adisplay 49, as illustrated in FIG. 1 . The video decoder 43, theviewpoint position detector 45, the viewpoint interpolator 47, and theoperation information adder 48 are included in a controller. Thecontroller may be implemented by dedicated hardware such as an ASIC or aFPGA, implemented by a processor, or implemented by a combination ofdedicated hardware and a processor. The viewing request transmitter 41,the video receiver 42, and the viewpoint information transmitter 46 areincluded in a communication interface.

The viewing request transmitter 41 transmits a viewing requestindicating content selected by the user of the receiving device 40, tothe second transmitting device 30.

The video receiver 42 receives video signals and metadata (positioninformation and ID information) from the second transmitting device 30.The video receiver 42 outputs the received video signals and metadata tothe video decoder 43 in association with each other.

The video decoder 43 decodes the video signals to generate decodedvideo, and outputs the decoded video to the storage 44.

The storage 44 stores the decoded video generated by the video decoder43.

The viewpoint position detector 45 detects the viewpoint position of theuser, and outputs viewpoint information indicating the detectedviewpoint position to the viewpoint information transmitter 46. Forexample, the viewpoint position detector 45 acquires an image of theuser's eyes captured by an imaging device included in the receivingdevice 40 and analyzes the image using any known technology, to detectthe viewpoint position. The viewpoint position detector 45 may be aline-of-sight sensor included in the receiving device 40 separately fromthe controller. The controller acquires the viewpoint position of theuser by the viewpoint position detector 45.

The viewpoint information transmitter 46 transmits the viewpointinformation detected by the viewpoint position detector 45, to thesecond transmitting device 30. The viewpoint information transmitter 46may transmit the viewpoint information to the second transmitting device30 at certain intervals, or transmit the viewpoint information to thesecond transmitting device 30 only in the case where the amount ofchange of the viewpoint position is greater than a threshold.

The viewpoint interpolator 47 performs viewpoint interpolation using aplurality of decoded videos (i.e. decoded videos obtained by decodingvideo signals corresponding to the position information determined bythe position determiner 36) having a common first group ID and secondgroup ID stored in the storage 44, to generate video corresponding tothe viewpoint position detected by the viewpoint position detector 45.Any known technology (for example, see PTL 1) may be used for viewpointinterpolation. The viewpoint interpolator 47 outputs theviewpoint-interpolated decoded video to the operation information adder48.

The operation information adder 48 acquires default decoded video fromthe storage 44, adds operation information to the decoded video, andoutputs them to the display 49, in response to a viewing request by theuser. After the viewpoint interpolation by the viewpoint interpolator47, the operation information adder 48 acquires theviewpoint-interpolated decoded video from the viewpoint interpolator 47,adds operation information to the decoded video, and outputs them to thedisplay 49.

Herein, “operation information” is information indicating, to the user,an operation method for changing the viewpoint position. For example,the operation information is information indicating an operation methodfor changing to a different viewpoint position in the same location orfor changing to a viewpoint position in a different location. Theoperation information may include an icon. The user can request a changeof the viewpoint position, by operating based on the operationinformation. For example, in the case where the receiving device 40 is ahead-mounted display, the user may perform the operation based on theoperation information by the direction of the line of sight. In thiscase, the line-of-sight direction is detected by the viewpoint positiondetector 45. In the case where the receiving device 40 is a tabletterminal, the user may perform the operation based on the operationinformation by screen operation such as flicking or swiping. In the casewhere the receiving device 40 is a non-mobile or non-portable devicehaving a flat panel display, the user may perform the operation based onthe operation information using a remote control.

The display 49 is, for example, a liquid crystal display or an organicelectroluminescent (EL) display. The display 49 displays the decodedvideo and the operation information received from the operationinformation adder 48.

FIG. 6 illustrates an example of display of video and operationinformation by the display 49. In the example illustrated in FIG. 6 ,video 481 and operation information 482 to 486 are displayed. Theoperation information 482 indicates that the currently displayed video481 is video based on a video group of first group ID=II in the locationC. The operation information 483 indicates that the viewpoint positionin the location C can be changed (i.e. the currently displayed video 481can be changed to video based on a video group of first group ID=I inthe location C) by a predetermined operation. The operation information484 indicates that the viewpoint position in the location C can bechanged (i.e. the currently displayed video 481 can be changed to videobased on a video group of first group ID=III in the location C) by apredetermined operation. The operation information 485 indicates thatthe viewpoint position can be changed to a different location (i.e. thecurrently displayed video 481 can be changed to video captured in thelocation A) by a predetermined operation. The operation information 486indicates that the viewpoint position can be changed to a differentlocation (i.e. the currently displayed video 481 can be changed to videocaptured in the location B) by a predetermined operation.

When the user operates based on operation information displayed by thedisplay 49, the viewpoint position detector 45 detects the viewpointposition corresponding to the operation. Thus, the user can request thesecond transmitting device 30 to change the viewpoint position.

(Operation Sequence)

The operation sequence of the video transmission system 1 according tothis embodiment will be described below, with reference to FIG. 7 . FIG.7 is a diagram illustrating the operation sequence between the secondtransmitting device 30 and the receiving device 40.

In step S101, the receiving device 40 transmits a viewing requestindicating content selected by the user to the second transmittingdevice 30, by the viewing request transmitter 41. The secondtransmitting device 30 receives the viewing request transmitted from thereceiving device 40, by the viewing request receiver 34.

In step S102, the second transmitting device 30 transmits, for thecontent requested for viewing by the receiving device 40, all IDinformation and the position information of the imaging device 10representative of each first-level group to the receiving device 40, bythe video transmitter 37. The receiving device 40 receives the foregoinginformation relating to the content requested for viewing from thesecond transmitting device 30, by the video receiver 42. For example, inthe case where the imaging devices 10 are installed as illustrated inFIG. 2 , the position information of the imaging device 10representative of each first-level group is the position information ofthe imaging device 10 located at the center of the first-level group,i.e. position information p=8, 23, 38. The video transmitter 37 maytransmit, for each first-level group, position information of each of aplurality of imaging devices 10 to the receiving device 40.

In step S103, the second transmitting device 30 transmits a defaultvideo signal to the receiving device 40 for the content requested forviewing by the receiving device 40, by the video transmitter 37. It isassumed here that the second transmitting device 30 sets a default videosignal for each content beforehand. For example, in the case where thevideo signals relating to the content requested for viewing are groupedas illustrated in FIG. 4 , the default video is video signal 8 of secondgroup ID=A and first group ID=I (video signal captured by the imagingdevice 10 of position information p=8 located at the center on thebehind the catcher side in FIG. 2 ).

In step S104, the receiving device 40 receives the default video signalof the content requested for viewing from the second transmitting device30, by the video receiver 42. The receiving device 40 decodes the videosignal by the video decoder 43, and displays the default video by thedisplay 49.

In step S105, the receiving device 40 transmits viewpoint information tothe second transmitting device 30, by the viewpoint informationtransmitter 46.

In step S106, the second transmitting device 30 specifies a request IDwhich is the ID of a video signal corresponding to a change of theviewpoint position of the user, by the position determiner 36. In thecase where the request ID is the same as the ID of the immediatelypreviously transmitted video signal, the second transmitting device 30advances the process to step S107. In the case where the request ID isdifferent from the ID of the immediately previously transmitted videosignal, the second transmitting device 30 advances the process to stepS109.

In step S107, the second transmitting device 30 transmits a plurality ofvideo signals necessary to generate video corresponding to the viewpointposition of the user to the receiving device 40, by the videotransmitter 37.

In step S108, the receiving device 40 performs viewpoint interpolationusing the plurality of video signals necessary for viewpointinterpolation received from the second transmitting device 30, by theviewpoint interpolator 47. The receiving device 40 displays theviewpoint-interpolated video by the display 49. The receiving device 40then returns the process to step S105.

In step S109, the second transmitting device 30 transmits, to thereceiving device 40, a default video signal in a video signal group towhich the request ID is assigned, by the video transmitter 37.

In step S110, the receiving device 40 receives the default video signalof the changed viewpoint position from the second transmitting device30, by the video receiver 42. The receiving device 40 decodes the videosignal by the video decoder 43, and displays the default video by thedisplay 49. The receiving device 40 then returns the process to stepS105.

As described above, the video transmission system 1 that transmits aplurality of video signals captured at different positions groups thevideo signals and assigns IDs. As a result of grouping the plurality ofvideo signals as first-level groups, a plurality of video signals usedfor viewpoint interpolation can be specified easily. As a result ofgrouping the plurality of video signals as second-level groups, possibleviewpoint position changes can be presented to the user. The user canthen view video of the viewpoint position changed within the samelocation or changed to a different location, by performing operationaccording to operation information displayed by the display 49.

Embodiment 2

A video transmission system according to Embodiment 2 will be describedbelow. FIG. 8 is a diagram illustrating a video transmission system 2according to Embodiment 2. The video transmission system 2 includes aplurality of imaging devices 10, a plurality of first transmittingdevices 20, a second transmitting device 30 a, and a receiving device 40a. The imaging devices 10 and the first transmitting devices 20 in thisembodiment are the same as the imaging devices 10 and the firsttransmitting devices 20 in Embodiment 1 respectively, and accordinglytheir description is omitted. The second transmitting device 30 a andthe receiving device 40 a in this embodiment are partly the same as thesecond transmitting device 30 and the receiving device 40 in Embodiment1 respectively, and the same components are given the same referencesigns and their description is omitted as appropriate.

The receiving device 40 a in this embodiment differs from the receivingdevice 40 in Embodiment 1 in that an ID information detector 50 isfurther included and a viewpoint information transmitter 46 a isincluded instead of the viewpoint information transmitter 46. The IDinformation detector 50 is included in the controller, too.

The ID information detector 50, in the case where the user performsoperation based on operation information displayed by the display 49 torequest a change of the viewpoint position, detects an ID (request ID)assigned to a video signal group corresponding to the new viewpointposition requested by the user. The ID information detector 50 outputsthe detected request ID to the viewpoint information transmitter 46 a asviewpoint information. For example, in the case where the user swipesalong the arrow icon indicated by the operation information 483 in FIG.6 , the ID information detector 50 detects first group ID=I and secondgroup ID=C as the request ID. In the case where the user swipes alongthe arrow icon indicated by the operation information 485 in FIG. 6 ,the ID information detector 50 detects first group ID=default value andsecond group ID=A as the request ID. In the case where the first groupID is a default value, the request ID may include only the second groupID.

The viewpoint information transmitter 46 a transmits the viewpointinformation detected by the viewpoint position detector 45 to the secondtransmitting device 30 a, in the case where the user does not performoperation based on operation information displayed by the display 49.The viewpoint information transmitter 46 a transmits the viewpointinformation (request ID) detected by the ID information detector 50 tothe second transmitting device 30 a, in the case where the user performsoperation based on operation information displayed by the display 49.

The second transmitting device 30 a in this embodiment differs from thesecond transmitting device 30 in Embodiment 1 in that a viewpointinformation receiver 35 a, a position determiner 36 a, and a videotransmitter 37 a are included instead of the viewpoint informationreceiver 35, the position determiner 36, and the video transmitter 37respectively.

The viewpoint information receiver 35 a, having received viewpointinformation not including a request ID, outputs the viewpointinformation to the position determiner 36 a, as in Embodiment 1. Theviewpoint information receiver 35 a, having received viewpointinformation including a request ID, specifies position informationcorresponding to a default video signal in a video signal group to whichthe request ID is assigned, and transmits the specified positioninformation to the video transmitter 37 a.

The position determiner 36 a, having received the viewpoint informationnot including a request ID from the viewpoint information receiver 35 a,outputs the respective position information of a plurality of imagingdevices 10 that capture a plurality of video signals necessary togenerate video corresponding to the viewpoint position of the user, tothe video transmitter 37 a. The position determiner 36 a, havingreceived the viewpoint information including a request ID from theviewpoint information receiver 35 a, outputs position informationcorresponding to a default video signal in a video signal group to whichthe request ID is assigned, to the video transmitter 37 a.

The video transmitter 37 a, having received the position informationfrom the viewpoint information receiver 35 a or the position determiner36 a, acquires a video signal captured by the imaging device 10 havingthe position information (video signal corresponding to the positioninformation) from the storage 33, and transmits the video signal to thereceiving device 40 a.

(Operation Sequence)

The operation sequence of the video transmission system 2 according tothis embodiment will be described below, with reference to FIG. 9 . FIG.9 is a diagram illustrating the operation sequence between the secondtransmitting device 30 a and the receiving device 40 a. Steps S201 toS204 are the same as steps S101 to S104 in Embodiment 1 described abovewith reference to FIG. 7 , and accordingly their description is omitted.

In step S205, the receiving device 40 a transmits viewpoint informationto the second transmitting device 30 a, by the viewpoint informationtransmitter 46 a.

In step S206, the second transmitting device 30 a determines whether theviewpoint information includes a request ID, by the position determiner36 a. In the case where the viewpoint information does not include arequest ID, the second transmitting device 30 a advances the process tostep S207. In the case where the viewpoint information includes arequest ID, the second transmitting device 30 a advances the process tostep S209.

Steps S207 to S210 are the same as steps S107 to S110 in Embodiment 1described above with reference to FIG. 7 , and accordingly theirdescription is omitted.

As described above, in this embodiment, the detection of a request ID isperformed not by the second transmitting device 30 a but by thereceiving device 40 a. In this case, too, the same effects as inEmbodiment 1 can be achieved.

Embodiment 3

A video transmission system according to Embodiment 3 will be describedbelow. FIG. 10 is a diagram illustrating a video transmission system 3according to Embodiment 3. The video transmission system 3 includes aplurality of imaging devices 10, a plurality of first transmittingdevices 20, a second transmitting device 30 b, and a receiving device 40b. The imaging devices 10 and the first transmitting devices 20 in thisembodiment are the same as the imaging devices 10 and the firsttransmitting devices 20 in Embodiment 1 respectively, and accordinglytheir description is omitted. The second transmitting device 30 b andthe receiving device 40 b in this embodiment are partly the same as thesecond transmitting device 30 and the receiving device 40 in Embodiment1 respectively, and the same components are given the same referencesigns and their description is omitted as appropriate.

In each of Embodiments 1 and 2 described above, a transmission path forbidirectional transmission is assumed as the transmission path betweenthe second transmitting device 30 or 30 a and the receiving device 40 or40 a. In this embodiment, on the other hand, a transmission path forunidirectional transmission is assumed as the transmission path betweenthe second transmitting device 30 b and the receiving device 40 b. Thetransmission path may be a transmission path supporting IP multicast, ora transmission path supporting broadcast waves. IP multicast through acommunication line may be transmission by broadcast waves. A broadcastsystem supporting IP packet transmission such as ISDB (IntegratedServices Digital Broadcasting) -S3 or ATSC (Advanced Television SystemsCommittee) 3.0 is considered to be a transmission path similar to IPmulticast through a communication line.

The second transmitting device 30 b in this embodiment differs from thesecond transmitting device 30 in Embodiment 1 in that the viewpointinformation receiver 35 and the position determiner 36 are not includedand a video transmitter 37 b is included instead of the videotransmitter 37.

In the case of performing IP multicast transmission through acommunication line, the video transmitter 37 b, having received aviewing request from the viewing request receiver 34, acquires all videosignals relating to content requested for viewing from the storage 33.The video transmitter 37 b then transmits the all video signalssimultaneously to the receiving device 40 b. Each video signal ismultiplexed with metadata (ID information and position information), asin the other embodiments. For example, in the case where the videosignals for the content requested for viewing are grouped as illustratedin FIG. 4 , the video transmitter 37 b transmits the 145 video signalssimultaneously to the receiving device 40 b.

In the case of performing broadcast transmission, there is no viewingrequest by the user, and the video transmitter 37 b always acquires allvideo signals relating to all content from the storage 33 withoutreference to a viewing request from the receiving device 40 b. The videotransmitter 37 b then transmits the all video signals relating to allcontent simultaneously to the receiving device 40 b. That is, content isselected in the receiving device 40 b. Hence, in the case of thebroadcast transmission path, the viewing request transmitter 41 and theviewing request receiver 34 may be omitted.

The receiving device 40 b in this embodiment differs from the receivingdevice 40 in Embodiment 1 in that the viewpoint information transmitter46 is not included, an ID information detector 50, a position determiner52, and a viewpoint changer 53 are further included, and a viewpointinterpolator 47 b and an operation information adder 48 b are includedinstead of the viewpoint interpolator 47 and the operation informationadder 48 respectively. The ID information detector 50, the positiondeterminer 52, and the viewpoint changer 53 are included in thecontroller, too.

The position determiner 52, having received viewpoint information fromthe viewpoint position detector 45, determines whether the viewpointposition changes. For example, in the case where the amount of change ofthe viewpoint position is greater than a threshold, the positiondeterminer 52 determines that the viewpoint position changes. In thecase where the position determiner 52 determines that the viewpointposition changes, the position determiner 52 specifies the respectiveposition information of a plurality of imaging devices 10 that capture aplurality of video signals necessary to generate video corresponding tothe viewpoint position of the user, and outputs the specified positioninformation to the viewpoint interpolator 47 b.

The viewpoint interpolator 47 b, having received the positioninformation from the position determiner 52, acquires decoded videocorresponding to the position information from the storage 44. Theviewpoint interpolator 47 b performs viewpoint interpolation using theacquired decoded video, and outputs the viewpoint-interpolated decodedvideo to the operation information adder 48 b.

The ID information detector 50, in the case where the user performsoperation based on operation information displayed by the display 49 torequest a change of the viewpoint position, detects an ID (request ID)assigned to a video signal group corresponding to the new viewpointposition requested by the user. The ID information detector 50 outputsthe detected request ID to the viewpoint changer 53.

The viewpoint changer 53, having received the request ID from the IDinformation detector 50, acquires default decoded video out of decodedvideo of a video signal group to which the request ID is assigned fromthe storage 44, and outputs the default decoded video to the operationinformation adder 48 b.

The operation information adder 48 b, during content playback, firstacquires default decoded video from the storage 44, adds operationinformation to the decoded video, and outputs them to the display 49.After the viewpoint interpolation by the viewpoint interpolator 47 b,the operation information adder 48 b acquires the viewpoint-interpolateddecoded video from the viewpoint interpolator 47 b, adds operationinformation to the decoded video, and outputs them to the display 49.After the detection of the request ID by the viewpoint changer 53, theoperation information adder 48 b acquires default decoded video of thechanged viewpoint position from the viewpoint changer 53, adds operationinformation to the decoded video, and outputs them to the display 49.

(Operation Sequence)

The operation sequence of the video transmission system 3 according tothis embodiment will be described below, with reference to FIG. 11 .FIG. 11 is a diagram illustrating the operation sequence between thesecond transmitting device 30 b and the receiving device 40 b in thecase of IP multicast transmission of video signals.

In step S301, the receiving device 40 b transmits a viewing requestindicating content selected by the user to the second transmittingdevice 30 b, by the viewing request transmitter 41. The secondtransmitting device 30 b receives the viewing request transmitted fromthe receiving device 40 b, by the viewing request receiver 34.

In step S302, the second transmitting device 30 b transmits all videosignals relating to the content requested for viewing by the receivingdevice 40 b to the receiving device 40 b together with metadata (IDinformation and position information), by the video transmitter 37 b.The receiving device 40 b receives the all video signals relating to thecontent requested for viewing together with the metadata from the secondtransmitting device 30 b, by the video receiver 42.

In step S303, the receiving device 40 b decodes the video signals by thevideo decoder 43. The receiving device 40 b then displays the defaultvideo of the content requested for viewing, by the display 49.

In step S304, the receiving device 40 b determines whether there is achange (minor change) of the viewpoint position not involving a changeof ID, by the position determiner 52. In the case where there is a minorchange of the viewpoint position, the receiving device 40 b advances theprocess to step S305. In the case where there is no minor change of theviewpoint position, the receiving device 40 b advances the process tostep S306.

In step S305, the receiving device 40 b performs viewpoint interpolationusing the plurality of decoded videos and generates video correspondingto the viewpoint position of the user, by the viewpoint interpolator 47b. The receiving device 40 b displays the viewpoint-interpolated videoby the display 49. The receiving device 40 b then returns the process tostep S304.

In step S306, the receiving device 40 b determines whether there is achange (major change) of the viewpoint position involving a change ofID, by the ID information detector 50. In the case where there is amajor change of the viewpoint position, the receiving device 40 bdetects a request ID, and advances the process to step S307. In the casewhere there is no major change of the viewpoint position, the receivingdevice 40 b returns the process to step S304.

In step S307, the receiving device 40 b acquires default decoded videoout of decoded video of a video signal group to which the request ID isassigned, by the viewpoint changer 53. The receiving device 40 b thendisplays the default video by the display 49.

In the case of broadcast transmission, the operation in step S301 is notperformed, and, in step S302, the second transmitting device 30 btransmits all video signals relating to all content to the receivingdevice 40 b together with metadata (ID information and positioninformation), by the video transmitter 37 b. Subsequently, in step S303,the receiving device 40 b displays default video of content selected bythe user. The operation from step S304 onward is the same as theoperation in the case of IP multicast transmission.

As described above, the second transmitting device 30 b transmits allvideo signals to the receiving device 40 b together with metadata (IDinformation indicating first-level grouping and second-level grouping,and position information). Therefore, in the case where the viewpointposition changes, the receiving device 40 b can specify necessarydecoded video depending on the change of the viewpoint position. Thus,the presently disclosed techniques can be applied to broadcasting or IPmulticast distribution of video signals.

<Program>

A computer capable of executing program instructions may be used tofunction as each of the foregoing first transmitting device 20, secondtransmitting device 30, 30 a, or 30 b, and receiving device 40, 40 a, or40 b. FIG. 12 is a block diagram illustrating the schematic structure ofa computer that functions as the first transmitting device 20, thesecond transmitting device 30, 30 a, or 30 b, or the receiving device40, 40 a, or 40 b. A computer 100 may be a general-purpose computer, aspecial-purpose computer, a workstation, a PC, an electronic notepad, orthe like. The program instructions may be program codes, code segments,or the like for executing necessary tasks.

The computer 100 includes a processor 110, a read only memory (ROM) 120,a random access memory (RAM) 130, a storage 140, an input unit(interface) 150, an output unit (interface) 160, and a communicationinterface (I/F) 170, as illustrated in FIG. 12 . The components arecommunicably connected to each other via a bus 180. The processor 110 isspecifically a central processing unit (CPU), a micro processing unit(MPU), a graphics processing unit (GPU), a digital signal processor(DSP), a system on a chip (SoC), or the like. The processor 110 may becomposed of a plurality of processors of the same type or differenttypes.

The processor 110 controls each component and performs variousarithmetic processes. In detail, the processor 110 reads a program fromthe ROM 120 or the storage 140, and executes the program using the RAM130 as a work area. The processor 110 controls each component andperforms various arithmetic processes according to the program stored inthe ROM 120 or the storage 140. In this embodiment, a program accordingto the present disclosure is stored in the ROM 120 or the storage 140.

The program may be recorded on a recording medium readable by thecomputer 100. By using such a recording medium, it is possible toinstall the program on the computer 100. The recording medium on whichthe program is recorded may be a non-transitory recording medium. Thenon-transitory recording medium is not limited, but may be, for example,a CD-ROM, a DVD-ROM, a USB (Universal Serial Bus) memory, or the like.The program may be downloaded from an external device via a network.

The ROM 120 stores various programs and various data. The RAM 130temporarily stores a program or data as a work area. The storage 140includes a hard disk drive (HDD) or a solid state drive (SSD), andstores various programs including an operating system and various data.

The input unit 150 includes one or more input interfaces for receivinginput operation by a user and acquiring information based on theoperation by the user. Non-limiting examples of the input unit 150include a pointing device, a keyboard, and a mouse.

The output unit 160 includes one or more output interfaces foroutputting information. Non-limiting examples of the output unit 160include a display for outputting information as images and a speaker foroutputting information as sound. In the case where the output unit 160is a touch panel-type display, the output unit 160 also functions as theinput unit 150.

The communication interface 170 is an interface for communicating withan external device.

Although the foregoing embodiments have been described as typicalexamples, it will be apparent to those skilled in the art that manychanges and substitutions can be made within the spirit and scope of thepresent disclosure. Therefore, the present disclosure should not beconstrued as being limited by the foregoing embodiments, and variousmodifications and changes can be made without departing from the claims.For example, a plurality of structural blocks or a plurality ofprocessing steps in the foregoing embodiments may be combined into onestructural block or one processing step, and one structural block or oneprocessing step in the foregoing embodiments may be divided into aplurality of structural blocks or a plurality of processing steps.

REFERENCE SIGNS LIST

-   -   1,2, 3 video transmission system    -   10 imaging device    -   20 first transmitting device    -   21 video receiver    -   22 video encoder    -   23 first ID assigner    -   24 storage    -   25 video transmitter    -   30, 30 a, 30 b second transmitting device    -   31 video receiver    -   32 second ID assigner    -   33 storage    -   34 viewing request receiver    -   35, 35 a viewpoint information receiver    -   36 position determiner    -   37 video transmitter    -   40, 40 a, 40 b receiving device    -   41 viewing request transmitter    -   42 video receiver    -   43 video decoder    -   44 storage    -   45 viewpoint position detector    -   46, 46 a viewpoint information transmitter    -   47, 47 b viewpoint interpolator    -   48, 48 b operation information adder    -   49 display    -   50 ID information detector    -   52 position determiner    -   53 viewpoint changer    -   100 computer    -   110 processor    -   120 ROM    -   130 RAM    -   140 storage    -   150 input unit    -   160 output unit    -   170 communication interface    -   180 bus    -   481 video    -   482 to 486 operation information

The invention claimed is:
 1. A transmitting device configured totransmit, to a receiving device, a plurality of video signals capturedfrom different positions, the plurality of video signals being groupedby a plurality of groups depending on imaging positions at which thevideo signals are captured, the transmitting device comprising: acontroller configured to assign an ID for identifying each of theplurality of groups, wherein the controller is further configured to:specify a request ID that is an ID of a video signal corresponding to achange of a viewpoint position of a user of the receiving device; and acommunication interface configured to transmit a video signal to whichthe ID is assigned, to the receiving device, the communication interfacefurther transmits the video signal to which the request ID is assigned,to the receiving device, wherein the communication interface is furtherconfigured to: transmit a plurality of video signals necessary togenerate video corresponding to the viewpoint position of the user in avideo signal group to which the request ID is assigned, when the requestID is same as an ID of an immediately previously transmitted videosignal; and transmit a default video signal in the video signal group towhich the request ID is assigned, when the request ID is different fromthe ID of the immediately previously transmitted video signal.
 2. Thetransmitting device according to claim 1, wherein the plurality of videosignals are grouped by the plurality of groups depending on the imagingpositions, and the controller is configured to assign a group ID foridentifying each of the plurality of groups.
 3. The transmitting deviceaccording to claim 2, wherein a first-level group out of the pluralityof groups is a group to which a plurality of video signals used togenerate video from a viewpoint different from an imaging position inthe receiving device belong.
 4. The transmitting device according toclaim 3, wherein a second-level group out of the plurality of groups isone or more first-level groups to which a plurality of video signalscaptured in a same location belong.
 5. The transmitting device accordingto claim 3, wherein the plurality of video signals belonging to thefirst-level group are compressed by predictive coding using acorrelation between the video signals, when transmitted by thecommunication interface.
 6. The transmitting device according to claim1, wherein the communication interface is configured to simultaneouslytransmit all of the plurality of video signals to each of which the IDis assigned, to the receiving device.
 7. The transmitting deviceaccording to claim 6, wherein the plurality of video signals are groupedby the plurality of groups depending on the imaging positions, and thecontroller is configured to assign a group ID for identifying each ofthe plurality of groups.
 8. The transmitting device according to claim7, wherein a first-level group out of the plurality of groups is a groupto which a plurality of video signals used to generate video from aviewpoint different from an imaging position in the receiving devicebelong.
 9. The transmitting device according to claim 8, wherein asecond-level group out of the plurality of groups is one or morefirst-level groups to which video signals captured in a same locationbelong.
 10. The transmitting device according to claim 8, wherein theplurality of video signals belonging to the first-level group arecompressed by predictive coding using a correlation between the videosignals, when transmitted by the communication interface.
 11. Thetransmitting device according to claim 1, wherein the ID and the videosignal are stored in a file of ISO base media file format.
 12. A systemcomprising: A transmitting device according to claim 1; A receivingdevice configured to receive a video signal transmitted from thetransmitting device according to claim 1, the receiving devicecomprising: a controller configured to decode the video signal togenerate decoded video, and acquire a viewpoint position of a user; anda communication interface configured to transmit viewpoint informationindicating the viewpoint position of the user, to the transmittingdevice.
 13. A system comprising: A transmitting device according toclaim 1; A receiving device configured to receive a video signaltransmitted from the transmitting device according to claim 1, togetherwith position information of an imaging device that captures the videosignal, the receiving device comprising a controller configured todecode the video signal to generate decoded video, and acquire aviewpoint position of a user, wherein the controller is configured tospecify respective position information of imaging devices that capturea plurality of video signals necessary to generate video correspondingto the viewpoint position of the user, and perform viewpointinterpolation using decoded video corresponding to the positioninformation.